A Walkthrough System to Display Video Corresponding to the Viewer’s Face Orientation

Walkthrough systems have hitherto been developed with the ability to produce a virtual display of a person’s field of view. If the actual position and orientation of a person’s face can be ascertained, then walkthrough images could be created to match this person’s face direction, thereby recreating video corresponding to the view from this person’s face direction. A possible application of this system is in store surveillance systems, where the direction in which someone is looking can be estimated to visualize what they are looking at in order to identify suspicious behavior. Therefore, we aim to produce a walkthrough system that uses image processing to acquire a person’s position and face direction, and displays video corresponding to this person’s face direction. In this system, multiple omnidirectional cameras are used to estimate the positions of people based on the bearings of moving bodies acquired by each camera. People’s face orientations are estimated from the skin regions of their faces. We have created a system that uses a person’s position and face orientation obtained in this way to provide an observer with walkthrough video corresponding to the person’s face orientation. To evaluate this system, we conducted experiments on the accuracy of human positions and face orientations and the accuracy of the output video, and we confirmed that the system can display video images corresponding to a person’s face direction.